This invention relates to the calibration of imaging systems, particularly for systems used for capturing colour information for meat or carcass sections for subsequent analysis.
There have been over recent years attempts to use more objective systems analysing meat sections such as carcasses in an abattoir for the purpose of grading. Attempts have been made to capture by means of colour cameras colour information for automated analysis.
Patent specification No. WO-91/14180 describes in some detail the background of the necessity and known practices to evaluate a carcass in an abattoir e.g. for predicting yield. That specification discloses the capture of an image and colour analysis to discriminate fat, meat, bruised tissue, etc. The specification recommends uniform controlled lighting conditions.
Patent specification WO-92/00523 discloses a carcass grading system using video cameras to capture images. This specification recognises the need for calibration to a defined set of environmental characteristics but merely observes that new parameters are needed if conditions change radically. The specification proposes that it is possible to have a xe2x80x9cdynamically self-calibratingxe2x80x9d system upon start up, and a system which is xe2x80x9cself-compensating in operationxe2x80x9d but without describing how.
Patent specification WO-95/21375 discloses a system for assessing meat pieces in an enclosure where there is controlled illumination and a controlled temperature. The system refers to the provision of reflection and colour references to enable adjustment of the system for instability in the camera or in the light sources.
Patent specification No. EP0730146 discloses a system using controlled illumination in which an image of the background is captured and from this an image of a carcass overlying the backgrounds is subtracted. The specification refers to the system being sensitive to camera variation and to the need for controlled illumination and to the consistency of position between the two images. Reference is made to the position of colour pads placed in the field of vision to enable adjustment of the camera, the lamps and/or RGB values.
All of these patent specifications gloss over the provisions for calibration, possibly because it was not recognised how critical calibration might be or alternatively the systems have not been developed sufficiently for the calibration problems to have been seriously addressed and adequate provisions made.
In developing a system to enable reliable prediction of the yield (or other characteristics) of a carcass, we have been attempting to rely on the recognition that colours of the carcass surface (which vary depending on whether there is fat or meat or selvedge exposed) are some of the best predictors of a carcass yield. We have been testing a system which uses the average colour or RGB values from a number of selected sections or xe2x80x9cpatchesxe2x80x9d on the surface of a carcass to predict the amount of fat cover automatically and, we have been using empirically obtained data to derive statistical equations relating the measured colour or RGB values to predict yield. Preferably multiple patches at different areas of the carcass are imaged, the selected patches being those which have been determined to have correlations with the actual bone out yield, and the system measures the average colour on each of these patches and these data are combined into the yield prediction equation. Patches with higher correlations with the actual bone out yield can have higher coefficients in the equation than those with a lower correlation. Basically the whiter the colour the more fat that the system infers.
Colour is difficult to measure absolutely. The colour seen by the human eye or by a video camera varies depending on the amount of and the colour of the light reflected from the viewed surface. We have found that whilst the absolute colour is not critical or essential to the successful operation of the yield prediction system, the relativity of the colour is desirably consistent from one plant or abattoir to another. If this is not the case, the system when used in different plants would predict different yields for identical carcasses which, of course, defeats the purpose of attempting to have a computerised or automated grading or classifying system. It follows therefore th at calibration of an imaging system for use in automated analysis of products such as meat carcasses becomes extremely important to provide consistent results in different environments of use.
Ac cordially it is an object of the present invention to provide a method and apparatus for calibration of an imaging system which can ensure or at least promote consistency of operation in different environments .
It is a further preferred object to provide a method and system for imaging having effective calibration provisions for the system to promote consistent results under different operating conditions.
According to a firsts aspect of the present invention there is provided a method for calibration of an imaging system having a camera for capturing colour data for a viewed scene, the method including a primary calibration to enable compensation for variations in the camera operation or characteristics, the primary calibration including providing a number of standard colour specimens and presenting these to the camera under controlled illumination conditions, capturing image data for the standard colours including colour data and calculating a primary transform comprising compensation factors or transformation parameters for relating the actual measured colour data to known standard colour data for the particular standard colour specimens; the method further including a secondary calibration to enable compensation for local lighting conditions during capture and processing of colour data for captured images to be analysed, the secondary calibration process comprising presenting to the camera reference colour specimens exposed to the lighting conditions that the object to be imaged and analysed is being or is to be exposed, followed by capturing image data for the reference colour specimens and determining a secondary transform comprising compensation factors or transformation parameters for relating the measured colour data for the reference colour specimens to known colour data for those specimens.
According to a second aspect of the present invention there is also provided a method for imaging a target object by an imaging system and providing corrected image data for the target object, the method including the steps of: calibrating the imaging system according to the first aspect; capturing image data for the target object; and applying the secondary transform and the primary transform to the image data for the target object so as to yield more objective colour data for the target object.
In use where the target object comprises a carcass to be graded or classified according to colour characteristics, the method of imaging preferably comprises locating the carcass at an image station adjacent to which are provided the reference colour specimens so that the carcass and reference colour specimens are imaged simultaneously under the same lighting conditions during the secondary calibration process.
The present invention also provides apparatus for calibrating an imaging system, the apparatus including means for performing the operations or steps of the method of the invention.
It will be convenient to describe the steps of the method and apparatus in conjunction with each other since separating the descriptions of the method and apparatus will be artificial and may make understanding more difficult. The invention however provides both a method and apparatus.
The primary calibration of present invention enables compensation for characteristics of the camera or cameras used in different environments to ensure a system used in one factory, abattoir, etc will not provide different results to a different camera because of different cam era response characteristics. Preferably, the primary calibration is carried out at the commencement of a sequence of imaging operations by the imaging system of successively presented target objects, the calculated primary transform for relating the actual measured colour data to known standard colour data for the particular standard colour specimens being stored for corrective processing of the colour data subsequently collected for successive target objects to be imaged and analysed.
In an abattoir, the primary calibration may be carried out relatively infrequently e.g. at commencement of daily operations of the system (preferably after the camera and standard light source have been powered up for a substantial time so as to be stabilised, e.g. for at least half an hour, and then possibly only a few more times for the rest of the day).
The secondary calibration i s carried out repeatedly throughout a period of use of the imaging system during which multiple target objects are sequentially presented for capture of image data useable for analysing the target objects, whereby time varying illumination conditions for target objects being sequentially presented can be compensated for in the captured image data for the target objects.
For example, the secondary calibration can be carried out at the time of capture of every image of an object to be analysed. The purpose of such continual calibration is to ensure reliability of image data processing in the presence of short term or intermittent variations in conditions such as ambient light variations due to time of day or due to normal operation of lighting equipment in a functioning abattoir, as well as power supply fluctuations to light sources, temperature variations, passing shadows, etc. The reference colour specimens used in the secondary calibration may comprise for example a display of multiple colour reference specimens presented simultaneously to the camera and being located in proximity to the location of the target object to be imaged and analysed so as to be exposed to substantially the same lighting conditions as the target object.
The secondary calibration process may include locating white reference specimen areas generally in the position that the target object to be imaged will occupy, followed by capturing image data for the white reference specimen areas and determining a spatial light level transform comprising compensating factors or transformation parameters to apply to measured colour data from the target object so as to compensate measured colour data for spatially varying light conditions at the position occupied by the target object. In one possible embodiment there are a number of white reference specimens dispersed over the area and preferably forwardly or in front of the position of the target object to be imaged, these white reference specimens being dispersed over the image area enabling the derivation of compensating factors or transformation parameters for spatially varying light conditions over the image area. In an alternative possible embodiment, wherein the white reference specimen areas are provided by a white board removably positioned at the site to be occupied by the target object to be imaged.
The apparatus of the present invention may provide some means for control of ambient influences, particularly in the case of the primary calibration. For this purpose, the primary calibration may include presenting the standard colour specimens to the camera under controlled illumination and with extraneous or external light being substantially excluded from illuminating the standard colour specimens.
Although the particular described carcass system is designed, constructed and operated to compensate for local ambient lighting variations, nevertheless some degree of ambient light control may be provided. For example, the target object to be imaged and analysed may be moved past screening means to the imaging position which the camera faces, and after imaging the object exits past further screening means. One advantage of this provision of screening means is to reduce the risk of false readings due for example to specular reflections at the target object from extraneous but transient light sources.
The method may further include a preliminary calibration in which the primary calibration standard colour specimens and associated controlled light source are calibrated against centralised standard colours by using the camera to capture image data for the centralised standard colours and determining a preliminary transform comprising compensation factors or transformation parameters for relating the actual measured colour data for the standard colour specimens to known standard colour data for the centralised standard colours. In the preferred embodiment of the process, the primary calibration standard colour specimen arrangement (i.e. the enclosure or hood, specimens and lighting) itself undergoes this preliminary calibration, e.g. in the controlled centralised environment (the xe2x80x9cLabxe2x80x9d) whereby the primary calibration standard colour specimens and associated controlled light source are calibrated against a centralised or laboratory standard, the calibration data for the primary standard colour specimens and associated light source being stored for use with a particular colour specimen and light source set. In this manner, when the primary standard colour specimen and associated light source set are used in the field, the colour data obtained can be related back to the centralised or laboratory standard, i.e. by the preliminary transform applied to the captured RGB data.
The present invention also provides in a further aspect an imaging system for use in imaging and analysis of a target object, the system including a camera for capturing colour data for the target object; a primary calibration means to enable compensation for variations in the camera operational characteristics, the primary calibration means including a number of standard colour specimens arranged to be presented to the camera under controlled illumination conditions, and means for calculating a primary transform comprising compensation factors or transformation parameters for relating the actual measured colour data provided by the camera to known standard colour data for the particular standard colour specimens; a secondary calibration means to enable compensation for local lighting conditions during capture and processing of colour data for a captured image of the target object, the secondary calibration means including reference colour specimens arranged to be presented to and imaged by the camera in a position so as to be exposed to the lighting conditions that the target object to be imaged and analysed is being or is to be exposed, and means for determining a secondary transform comprising compensation factors or transformation parameters for relating the measured colour data derived from the camera for the reference colour specimens to known colour data for those specimens.
In an imaging system according to the invention the primary calibration means may include an enclosure or hood arranged to be fitted to the camera and having the standard colour specimens therein illuminated by an internal light source, the enclosure shielding the standard colour specimens from external illumination, the primary transform being calculated to transform source colour data provided by the camera (which is located at the site where the target objects will be imaged) with the standard colour specimens being viewed within the enclosure into target colour data which is stored colour data for the standard colour specimens viewed by the same camera within the same enclosure in a controlled environment.
It will be convenient to further describe the method and apparatus of the present invention with particular reference to a system developed for imaging and analysing meat carcasses, particularly for colour analyses of whole carcasses to predict yield. However it will be appreciated that the method and apparatus are applicable to other imaging systems where colour data for imaged target objects are to be analysed and consistency of the analysis results between different imaging systems is desirable.